Yale  University 

Department  of  Computer  Science 


Robustness  of  Class-Based  Path- Vector  Systems 

Aaron  D.  laggard  Vijay  Ramachandran 

YALEU/DCSArR-1296 
December  2004 


This  work  was  partially  supported  by  the  U.S.  Department  of  Defense  (DoD)  University  Research  Initiative 
(URI)  program  administered  by  the  Office  of  Naval  Research  (ONR).  A  shortened  form  of  this  work  has 
been  published  as  a  conference  paper  [10]. 


Report  Documentation  Page 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 

1.  REPORT  DATE 

DEC  2004 

2.  REPORT  TYPE 

3.  DATES  COVERED 

00-12-2004  to  00-12-2004 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

Robustness  of  Class-Based  Path-Vector  Systems 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Yale  University, Department  of  Computer  Science, PO  Box  208285, New 
Haven, CT, 06520-8285 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 

OF  PAGES 

18 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Standard  Form  298  (Rev.  8-98} 

Prescribed  by  ANSI  Std  Z39-18 


Robustness  of  Class-Based  Path- Vector  Systems 


Aaron  D.  laggard* * 


Abstract 


Griffin,  Jaggard,  and  Ramachandran  [5]  introduced  a 
framework  for  studying  design  principles  for  path-vector 
protocols,  such  as  the  Border  Gateway  Protocol  (BGP) 
used  for  inter-domain  routing  in  the  Internet.  They  out¬ 
lined  how  their  framework  could  describe  Hierarchical- 
BGP-like  systems  in  which  routing  at  a  node  is  deter¬ 
mined  by  the  relationship  with  the  next-hop  node  on  a 
path  (e.g.,  an  ISP-peering  relationship)  and  some  addi¬ 
tional  scoping  rules  (e.g.,  the  use  of  backup  routes).  The 
robustness  of  these  class-based  path-vector  systems  de¬ 
pends  on  the  presence  of  a  global  constraint  on  the  sys¬ 
tem,  but  an  adequate  constraint  has  not  yet  been  given  in 
general.  In  this  paper,  we  give  the  best-known  sufficient 
constraint  that  guarantees  robust  convergence.  We  show 
how  to  generate  this  constraint  from  the  design  specifica¬ 
tion  of  the  path-vector  system.  We  also  give  centralized 
and  distributed  algorithms  to  enforce  this  constraint,  dis¬ 
cuss  applications  of  these  algorithms,  and  compare  them 
to  algorithms  given  in  previous  work  on  path-vector  pro¬ 
tocol  design. 
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Vijay  Ramachandran^ 

1  Introduction 

The  standard  Internet  inter-domain  routing  protocol,  the 
Border  Gateway  Protocol  (BGP),  determines  routes  using 
independently  configured  policies  in  autonomously  ad¬ 
ministered  networks.  Little  global  coordination  of  poli¬ 
cies  takes  place  between  domains,  or  autonomous  sys¬ 
tems  (ASes),  because  (1)  ASes  are  reluctant  to  reveal  de¬ 
tails  about  internal  routing  configuration,  and  (2)  BGP 
contains  no  reliable  mechanism  to  permanently  attach  in¬ 
formation  to  a  route  as  it  is  shared  throughout  the  net¬ 
work.  However,  without  global  coordination,  interaction 
of  locally  configured  policies  can  lead  to  global  routing 
anomalies  [1,  4,  7,  11,  13],  e.g.,  route  oscillation  and 
inconsistent  recovery  from  link  failures.  Because  the 
techniques  used  to  configure  policies  and  the  protocol’s 
specification  have  evolved  separately,  there  is  an  inherent 
trade-off  between,  on  one  hand,  maintaining  rich  semantic 
expressiveness  available  with  current  vendor-developed 
configuration  languages  and  autonomy  in  policy  config¬ 
uration,  and,  on  the  other  hand,  guaranteeing  that  the 
protocol  will  converge  robustly,  i.e.,  predictably,  even  in 
the  presence  of  link  and  node  failures.  Griffin,  Jaggard, 
and  Ramachandran  [5]  showed  that  achieving  all  three  of 
these  design  goals  requires  a  non-trivial  global  constraint 
on  the  network,  but  they  left  open  the  question  of  how  to 
identify  and  enforce  the  constraint. 

This  paper  answers  this  question  in  the  context  of  class- 
based  path-vector  systems.  Path- vector  systems,  intro¬ 
duced  in  [5],  provide  a  formal  model  for  design  and  analy¬ 
sis  of  path-vector  protocols  and  their  policy-configuration 
languages.  Class-based  systems  focus  on  a  generaliza¬ 
tion  of  next-hop-preference  routing,  where  routing  policy 
for  an  AS  can  be  defined  by  the  relationships  (commer¬ 
cial  or  otherwise)  between  it  and  its  neighboring  ASes. 
The  canonical  example  of  such  a  system  is  a  simplified 
version  of  BGP  that  takes  into  account  the  economic  re- 
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alities  of  today’s  commercial  Internet — that  ASes  parti¬ 
tion  their  neighbors  into  customers,  providers,  and  peers 
and  that  there  are  preference  guidelines  used  to  decide  be¬ 
tween  routes  learned  from  neighbors  of  different  classes. 
The  scope  of  class-based  systems,  however,  goes  beyond 
this  “Hierarchical  BGP”  system;  the  framework  can  also 
be  used  to  build  and  analyze  systems  with  complete  au¬ 
tonomy  and  those  that  allow  arbitrary  next-hop  prefer¬ 
ence  routing.  Furthermore,  any  protocol  specification  that 
can  be  described  by  a  countable-weight,  monotone  path- 
vector  algebra  [12]  can  also  be  described  by  a  class-based 
path-vector  system  [9]. 

In  this  paper,  we  provide  the  best  known  robustness 
constraint  for  class-based  systems.  The  constraint  ensures 
the  robustness  of  networks  that  satisfy  it;  it  is  in  fact  the 
best  possible  robustness  guarantee  because,  in  networks 
that  do  not  satisfy  it,  some  set  of  nodes  may  write  poli¬ 
cies  that  cause  route  oscillations.  (Our  proof  of  this  con¬ 
structs  such  policies.)  We  give  an  algorithm  to  generate 
the  constraint  given  only  the  design  specification  of  the 
system.  We  then  provide  centralized  and  distributed  al¬ 
gorithms  to  check  networks  for  violation  of  the  constraint 
and  discuss  their  applications,  including  how  to  use  our 
results  to  check  a  network  with  arbitrary  next-hop  prefer¬ 
ences  for  potential  bad  interactions.  The  distributed  algo¬ 
rithm  reveals  almost  no  private  policy  information,  pro¬ 
vides  several  options  for  correcting  a  constraint  violation, 
and  has  constant  message  complexity  per  link  and  limited 
storage  at  each  node.  We  compare  and  contrast  our  algo¬ 
rithms  with  those  in  previous  work. 

Although  it  may  be  sufficient  to  provide  a  supplemen¬ 
tary  protocol  enforcing  some  global  conditions  (and,  in¬ 
deed,  the  distributed  algorithm  in  this  paper,  modified  for 
BGP,  can  be  run  alongside  BGP  to  detect  potentially  bad 
policy  interactions),  there  are  several  benefits  to  this  ap¬ 
proach  of  analyzing  robust  protocol  convergence  from  a 
design-framework  perspective.  First,  the  algorithms  in 
our  paper  preclude  all  policy-based  oscillations  in  ad¬ 
vance;  as  long  as  the  constraint  is  enforced,  the  protocol 
can  safely  run  on  any  network.  Second,  the  approach  is  an 
integral  part  of  designing  policy-configuration  languages. 
The  design  framework  identifies  provably  sufficient  local 
and  global  conditions  needed  for  a  protocol  to  achieve 
its  design  goals.  Our  paper  precisely  gives  the  trade-off 
between  the  strength  of  local  policy  guidelines  built  into 
the  policy-configuration  language  and  the  strength  of  the 


global  assumption  needed  in  the  broad  class-based  con¬ 
text.  The  designer  can  use  these  results  to  consider  what 
balance  between  local  and  global  enforcement  is  desired 
and  can  incorporate  the  guidelines  generated  by  the  re¬ 
sults  in  this  paper  into  the  design  of  multiple  high-level 
policy  languages — all  before  running  the  protocol  on  an 
actual  network. 

1.1  Related  Work 

Several  papers  have  presented  policy-configuration  guide¬ 
lines  or  design  principles  to  prevent  global  routing  anoma¬ 
lies  in  path-vector  routing.  Gao  and  Rexford  [3]  showed 
that  route  preferences  and  scoping  consistent  with  the 
Hierarchical-BGP  example  mentioned  above  give  stable 
path-vector  routing  without  global  coordination.  Griffin, 
Shepherd,  and  Wilfong  [6]  defined  an  abstraction  of  path- 
vector  routing  and  identified  a  sufficient  condition  for  lo¬ 
cal  policies  that  prevents  policy-induced  route  oscillation; 
Griffin  and  Wilfong  [8]  used  those  conditions  to  give  a 
simple  path-vector  routing  algorithm  that  dynamically  de¬ 
tects  policy-induced  route  oscillations.  In  addition,  Gao, 
Griffin,  and  Rexford  [2]  combined  the  results  from  these 
papers  to  modify  a  simplified  version  of  BGP  to  perform 
stable  back-up  routing. 

Building  on  this  work,  the  papers  by  Griffin,  laggard, 
and  Ramachandran  [5]  and  Sobrinho  [12]  developed  for¬ 
mal  models  for  path-vector  routing  so  that  properties  of 
path-vector  protocols  can  be  studied  without  involving  de¬ 
tails  of  protocol  dynamics  or  of  actual  networks.  Both 
papers  prove  results  about  protocol  convergence  based 
on  the  design  specification  of  the  protocol  itself.  They 
present  an  application  of  their  frameworks  that  general¬ 
izes  systems  like  Hierarchical  BGP  (mentioned  above), 
incorporating  the  policy  guidelines  from  [2,  3,  6]  into  the 
design  of  protocol  systems.  This  paper  completes  the 
analysis  of  this  application;  It  answers  the  questions  left 
open  in  [5]  and  gives  results  that  can  be  used  in  the  general 
design  of  protocols  and  policy-configuration  languages. 

2  The  Class-Based  Framework 

Class-based  systems,  which  generalize  Hierarchical- 
BGP-like  systems,  are  a  type  of  globally  constrained 
path-vector  policy  systems  (PVPSes).  A  PVPS  describes 
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a  protocol  that  nodes  use  to  exchange  path  descriptors  (in¬ 
stances  of  the  protocol’s  data  structure  describing  routes 
in  the  network).  A  PVPS  specifies  a  ranking  function  oj 
from  the  set  TZ  of  all  path  descriptors  into  some  totally  or¬ 
dered  set;  if  r  and  r'  describe  two  different  paths  from  a 
node  u  to  a  network  destination,  v  will  prefer  the  one  de¬ 
scribed  by  r  if  uj{r)  <  ic{r').  The  PVPS  also  defines  al¬ 
lowable  import  and  export  policy  functions  that  nodes  use 
to  modify  path  descriptors  in  order  to  affect  their  rank; 
these  are  usually  compiled  from  policies  written  by  net¬ 
work  operators  at  each  node  in  some  high-level  policy  lan¬ 
guage.  We  will  recall  other  relevant  definitions  and  PVPS 
results  as  we  proceed;  for  a  fuller  exposition  of  PVPSes 
see  [5]. 

In  a  class-based  system,  a  path  descriptor  r  is  a  quadru¬ 
ple  {d,g,l,P),  where  d  is  the  forwarding  destination  of 
the  path  and  P  is  the  path  itself  The  attributes  g  and  I 
(whose  values  in  the  descriptor  r  are  denoted  g{r)  and 
l{r))  are  the  level  and  local  preference,  respectively,  of 
the  route;  these  take  values  in  N.  Level  is  shared  between 
nodes  but  local  preference  is  not.  The  ranking  function 
depends  on  these  attributes;  oj{r)  =  {g{r),—l{r)),  with 
lexicographic  ordering  on  the  values  of  w  so  that  r  will  be 
preferred  over  r'  if  g{r)  <  g{r')  or  if  g{r)  =  g{r')  and 
l{r)  >  l{r').  Policies  in  class-based  systems  may  filter 
routes,  weakly  increase  the  level  attribute,  and  change  the 
local  preference  as  desired,  but  nothing  else. 

Class-based  systems  incorporate  global  relative  prefer¬ 
ence  and  scoping  rules  using  these  two  attributes;  the  for¬ 
mer  affect  relative  rank  of  paths  imported  from  different 
classes  of  neighbors,  while  the  latter  specify  the  classes  to 
which  certain  routes  may  be  exported.  In  BGP,  it  is  com¬ 
mon  practice  today  for  a  router  to  always  prefer  a  route 
learned  from  a  customer  router  over  a  route  (to  the  same 
destination)  learned  from  a  peer  router  and  for  a  router  to 
always  export  routes  learned  from  its  customer(s)  to  its 
provider(s).  The  first  is  an  example  of  a  relative  prefer¬ 
ence  rule,  the  second  of  a  scoping  rule. 

A  class-based  PVPS  is  almost  entirely  described  by  a 
class  description  CD  =  (C,  X,  W,  M),  which  contains 
a  set  of  c  classes  and  three  c  x  c  matrices  that  describe  the 
relationship,  preference,  and  scoping  rules.  The  classes 
in  C  =  {Cl,  C2,  . . . ,  Cc},  e.g.  “customer”  or  “peer,” 
are  used  to  describe  the  relationship  between  one  node 
and  another.  In  an  instance  of  a  class-based  system,  each 
node  V  will  assign  a  class  C”  (it)  to  each  of  its  neighbors 


It;  C"  (it)  indicates  how  v  views  the  relationship  between 
nodes  it  and  v.  The  cross-class  matrix  X  describes  the 
allowable  ways  that  it  may  view  u’s  role  in  their  relation¬ 
ship  given  that  v  views  it’s  role  as  C'^{u).  A  necessary 
condition  for  an  instance  to  be  legal  is  that  for  every  two 
adjacent  nodes  it  and  v  in  the  network,  if  C'"{u)  =  Ci  and 
C^{v)  =  Cj,  then  Xij  =  1.  Without  loss  of  generality  we 
may  view  X  as  a  symmetric  matrix  {X  may  be  replaced 
with  the  matrix  {min{Xij ,  Xji))ij  without  changing  the 
legality  of  any  class-based  instance).  Note  that  if  C'"{u) 
always  uniquely  determines  the  value  of  C“(ii)  allowed 
by  X,  then  X  has  at  most  one  1  in  each  row  and  column; 
assuming  no  classes  are  superfluous,  X  is  then  a  permu¬ 
tation  matrix. 

Relative  preference  rules  are  described  by  the  ma¬ 
trix  W,  which  has  entries  from  the  set  (•)  = 

{<,<,=,>,>,T}  of  binary  comparison  operators, 
where  xT y  for  every  x,  y.  (We  will  refer  to  a  generic 
operator  in  (•)  with  the  symbol  •.)  If  a  node  v  has  im¬ 
ported  path  descriptors  and  rj  from  neighbors  whom 
V  views  as  being  in  classes  Ci  and  Cj,  respectively,  if 
•  =  Wij,  and  \f  g{ri)  =  g{rj),  then  the  local  preference 
values  l{ri)  and  l{rj)  must  be  set  (via  u’s  import  policy 
functions)  so  that  U!{ri)  •  u’(rj).  We  assume  that  the  pref¬ 
erence  relations  between  classes  specified  by  W  are  con¬ 
sistent  with  a  partial  order  on  C;  e.g.,  W  should  not  be 
such  that  W12  =  W23  =  W31  =  ‘<’. 

Therefore,  W  can  be  used  to  give  a  partial  order¬ 
ing  on  the  classes  describing  which  class  of  neighbor’s 
routes  should  be  preferred.  For  example,  given  the  above- 
mentioned  practice  of  preferring  customer  routes  to  peer 
routes,  we  would  set  Wij  =  “<’  where  Ci  is  the  class 
“customer”  and  Cj  is  the  class  “peer.”  Note  that  W  is 
an  antisymmetric  matrix  of  sorts  (because  the  compar¬ 
isons  are  antisymmetric  relations,  if  Wij  =  ‘<’  then 
Wji  =  ‘>’)  and  that  the  only  viable  setting  on  the  di¬ 
agonal  is  Wii  =  T.' 

The  scoping  rules  in  a  class-based  system  are  described 
by  M,  a  c  X  c  matrix  with  entries  from  (.)U{L}.  If 
Mij  =  _L,  then  v  may  not  export  path  descriptors  learned 
from  a  neighbor  in  class  Ci  to  a  neighbor  in  class  Cj. 

^The  setting  Wa  =  ‘=’  is  also  possible  but  not  viable.  In  this  case, 
all  descriptors  with  the  same  level  imported  from  a  neighbor  of  class 
Ci  would  have  to  have  equal  rank,  and  then  ties  could  not  be  broken  by 
changing  the  local  preference  based  on,  e.g.,  next-hop  address  or  shortest 
path  length. 
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If  Mij  =  •  7^  _L,  then  v  may  export  a  path  descrip¬ 
tor  learned  from  a  neighbor  in  class  Ci  to  one  in  class 
Cj ;  however,  if  is  the  descriptor  that  v  receives  (after 
applying  the  import  policy  function)  from  a  neighbor  in 
class  Ci,  v’s  export  policy  function  )  for  a  neighbor 

u  in  class  Cj  must  satisfy  g{ri)  •  g  ;  e.g.,  if 

Mij  =  C‘"{u)  =  i,  and  C'"{w)  =  j,  then  v  may 

export  routes  learned  from  u  to  w,  but  the  export  policy 
function  must  strictly  increase  the  level  attribute  of 

these  descriptors. 

In  our  analysis  here,  we  are  primarily  interested  in 
whether  or  not  the  entries  of  W  and  M  allow  equality. 
We  thus  define  two  {0,  l}-matrices  W  and  M  as  follows. 

Definition  2.1  ({0, 1} -Description  Matrices). 

1.  Let  Wij  be  0  if  Wij  permits  equality  (including  T). 
Let  Wij  =  =  “<’  and  let  Wj  =  1  if 

W.,  =  ‘>’. 


a  level  increase;  peer  routes  may  be  shared  with  peers 
and  providers  if  the  level  is  increased,  and  provider 
routes  may  be  shared  with  peers  if  the  level  is  increased. 
Provider  routes  may  not  be  shared  with  other  providers 
(as  no  node  should  carry  transit  traffic  between  two  of 
its  providers).  The  class-description  components  for  this 
system  are  thus: 
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The  corresponding  {0, 1} -matrices  are: 
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2.  Let  Mij  be  1  if  permits  equality,  i.e.,  Mij  = 
‘<’,  etc.,  and  let  Mij  be  0  otherwise. 

Example  2.2  (Hierarchical  BGP  with  Back-up 
Routes).  This  is  the  canonical  example  of  a  class-based 
system,  motivated  by  the  design  guidelines  in  [2,  3].  It 
uses  three  classes,  corresponding  to  standard  Internet 
business  relationships:  Ci  or  “customer” — someone 
to  whom  connectivity  is  sold;  C2  or  “peer” — someone 
with  whom  an  agreement  is  established  to  transit  traffic, 
usually  to  shortcut  more  expensive  or  long  routes;  and 
C3  or  “provider” — someone  from  whom  connectivity 
is  purchased.  The  class  consistency  rules  require  that 
C^(u)  =  C2  implies  C^(v)  =  C2,  and  C^{u)  =  Ci  iff 
C“(t;)  =  Ca.  The  level  attribute  is  used  to  mark  routes 
that  are  for  backup  use;  this  is  done  by  increasing  the 
level  on  export.  Because  this  is  a  shared,  nondecreasing 
attribute  that  has  precedence  in  ranking,  such  a  back-up 
route  will  not  be  selected  if  there  are  routes  with  lower 
level  available.  Following  economically  motivated  prac¬ 
tice,  customer  routes  {i.e.,  routes  learned  from  customers) 
are  preferred  to  peer  routes  when  their  level  attributes 
are  equal;  both  are  preferred  to  provider  routes.  The 
scoping  rules  allow  customer  routes  to  be  shared  with  all 
neighbors  (without  requiring  a  level  increase).  Peer  and 
provider  routes  may  be  shared  with  customers  without 


Omitting  the  ability  to  mark  routes  for  backup  use  (and 
generally  ignoring  the  level  attribute)  yields  the  simpler 
system  of  Hierarchical-BGP;  both  of  these  systems  are 
discussed  in  [5]. 

Remark  2.3.  A  proof  that  this  system  robust  (with  the 
additional  global  constraint  that  there  are  no  customer- 
provider  cycles)  can  be  found  in  [5].  We  will  return  to 
this  example  to  show  that  our  more  general  results  regard¬ 
ing  robustness  are  consistent  with  what  is  already  known 
about  these  systems. 

3  Robustness  and  Global 
Constraints 

Having  reviewed  the  class-based  framework,  we  now 
present  the  contributions  of  this  paper  in  detail.  In  this 
section,  we  motivate  and  describe  the  constraint  that  guar¬ 
antees  robust  convergence  for  protocols  modeled  by  the 
framework;  we  prove  its  correctness  and  show  how  to 
generate  it  given  only  the  class-based  design  specification 
of  a  protocol.  Then  we  show  how  to  check  this  constraint 
in  two  ways:  by  using  a  centralized  algorithm  (Sec.  4), 
and  by  using  a  distributed  algorithm  (Sec.  5)  that  addi¬ 
tionally  fixes  detected  constraint  violations. 
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3.1  Robustness  and  Dispute  Wheels 

The  major  goal  of  a  routing  system  is  the  assurance  that 
every  instance  of  a  network  and  local  policies  will  be 
robust,  i.e.,  that  every  legal  instance  of  the  system  will 
have  a  unique  routing  solution  as  will  all  sub-instances 
(these  might  arise  due  to  the  failure  of  any  set  of  links 
and  nodes).  The  additional  design  goals  of  autonomy, 
transparency,  and  expressiveness  were  formally  defined 
for  PVPSes  (and  thus  class-based  systems)  in  [5].  One  of 
the  major  results  in  that  paper  (Thm.  8,  [5])  is  that  any 
robust,  reasonably  autonomous,  transparent,  and  moder¬ 
ately  expressive  PVPS  must  have  a  global  constraint — a 
predicate  defined  on  instances  of  the  PVPS  whose  truth 
value  on  an  instance  determines  whether  that  instance  is 
“legal” — that  is  not  identically  true.  Class-based  systems, 
by  their  definition,  satisfy  these  design  goals  of  PVPSes; 
thus  they  must  include  a  non-trivial  global  constraint  if 
they  are  to  be  robust.  A  sufficient  global  constraint  for 
the  Hierarchical-BGP  systems  in  Ex.  2.2  is  that  the  sig¬ 
naling  graph  does  not  contain  any  customer-provider  cy¬ 
cles  {i.e.,  cycles  in  which  each  node  views  one  of  its 
neighbors  in  the  cycle  as  a  customer  and  the  other  as  a 
provider)  [3].  Our  goal  here  is  to  find  good  global  con¬ 
straints  for  class-based  systems  in  general.  As  a  starting 
point,  we  use  the  broadest-known  condition  for  robust¬ 
ness  of  path-vector  systems,  that  of  dispute-wheel  free¬ 
ness-,  this  was  presented  in  [6]  and  was  then  adopted  into 
the  PVPS  framework  in  [5].  We  now  turn  to  a  simpli¬ 
fied  definition  of  this  condition,  including  just  the  details 
needed  for  class-based  systems,  after  some  notation. 

Definition  3.1  (Descriptor  for  a  Realizable  Path).  Let 

Pn  =  VnVn-1  •  •  •  Uq  be  a  putative  route  in  a  path-vector 
instance,  such  that  has  learned  about  from  its  neigh¬ 
bor  Vn-i,  who  learned  about  P„_i  =  Vn-iVn-2  ■  ■  -  fo 
from  its  neighbor  Vn-2,  etc.  This  process  of  route  ex¬ 
change  began  with  node  uq  originating  a  path  descriptor 
tq  and  exporting  it  to  vi.  At  each  step  along  the  way,  a 
subpath  Pi  of  Pn  was  imported  at  node  Vi.  We  call  this  a 
realizable  path  because  it  can  be  selected  as  a  valid  for¬ 
warding  route  from  Vi. 

Given  an  edge  {m,  u}  in  the  network,  we  define  the 
arc-policy  function  for  the  signaling  edge  (u,  v)  to  be 
the  application  of  it’s  export  policy  function  P°^n) 

V  followed  by  the  application  of  v’s  import  policy  func¬ 
tion  F^nn)  Bsfors  the  descriptor  is  imported  at  v. 


the  protocol  updates  the  path  attribute  to  reflect  the  added 
edge  (u,  v),  filters  out  any  paths  which  contain  loops,  and 
hides  the  local-preference  value  used  by  u.^  Formally,  if 
i?  is  a  set  of  path  descriptors  known  to  u,  then  the  arc- 
policy  function  for  the  signaling  edge  {u,  v)  applied  to  R 
is 

I  {d,g,l,P)  G  uP  is  a  simple  path}); 

this  is  the  set  of  path  descriptors  that  v  receives  along  the 
edge  {u,  v).  Using  arc -policy  functions,  we  can  itera¬ 
tively  define  the  path  descriptor  associated  with  path  Pn 
and  its  subpaths,  assuming  that  it  originated  at  vq  with 
path  descriptor  tq.  Let 

r{Po)  =  ro, 

and  for  i,  1  <  i  <  n,  let 

r{Pi)  =  f{v,,v,.^){r{P^-l))■ 

Definition  3.2  (Dispute  Wheel).  An  instance  of  a  path- 
vector  system  contains  a  dispute  wheel  (see  Fig.  1)  if  there 
is  a  set  of  nodes  {uq,  vi, . . . ,  Vn]  C  V  such  that: 

1.  node  Vq  advertises  a  destination  and  nodes 
vj, ...  ,Vn  attempt  to  find  routes  to  that  desti¬ 
nation; 

2.  there  exists  a  signaling  path  Qi  for  each  i,  1  <  i  <  n, 
from  uq  to  Vi  (the  Qi  are  not  necessarily  disjoint);  let 
Qo  —  Qn^ 

3.  there  exists  a  signaling  path  Ri  from  each  Vi  to  Vi+i 
for  alH,  1  <  i  <  n  (where  Vn+i  =  vi,  the  Ri  are 
not  necessarily  disjoint);  let  i?o  =  Rn\ 

4.  Qi  and  Ri-iQi-i  are  realizable  paths  from  node  Vi 
to  Vq  for  all  i,  1  <  z  <  n;  and 

5.  uj{r{Ri-iQi-i))  <  uj(r(Qi))  for  alH,  1  <  i  <  n; 
i.e.,  every  node  Vi  prefers  to  reach  vq  by  using  the 
path  Ri-iQi-i  through  Vi-i  instead  of  the  path  Qi. 

^In  the  general  PVPS  framework,  policy  application  is  explicitly  de- 
fined  using  policy  application  functions',  for  class-based  systems,  these 
functions  are  responsible  for  the  updating  and  filtering  as  well  as  apply¬ 
ing  the  policies  in  the  manner  described. 
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The  paths  Qi  are  spokes  of  the  dispute  wheel,  the  paths 
Ri  constitute  the  rim  of  the  wheel,  and  the  nodes  on  the 
rim  of  the  wheel  are  its  rim  nodes. 

The  importance  of  dispute  wheels  is  shown  by  the  fol¬ 
lowing  theorem. 

Theorem  3.3  (Sec.  V,  [6]  and  Thm.  5,  [5]).  Any  dispute- 
wheel-free  PVPS  instance  is  robust. 

An  instance  of  a  PVPS  is  a  network  on  which  the  PVPS 
protocol  will  run,  combined  with  nodes’  import  and  ex¬ 
port  policies  that  are  permitted  by  the  PVPS  specification. 
Dispute  wheels  in  instances  of  class-based  systems  have 
the  following  property,  which  we  will  use  to  detect  and 
avoid  dispute-wheels  in  this  work. 

Proposition  3.4  (Lemma  8.5,  extended  version  of  [5]). 

The  level-attribute  values  of  all  the  path  descriptors  r(Qi) 
and  r{RiQj)  involved  in  a  dispute  wheel  are  equal. 

The  following  notation  and  conventions  make  it  easier 
to  discuss  class  assignments  on  a  dispute  wheel. 

Definition  3.5  (Edge-Class  Notation). 


;■  C™(m)  =a  \ 

[  c“(u)  =  c  .i 

V 

:  C’'(®)=7  i 

O" 


C^{w)=p 
c-[v)  =  n 


'6 


X 


■  •  ’  Dispute 
Wheel 


Figure  1:  Active  node  u  of  a  dispute  wheel. 


3.2  Generating  a  Global  Constraint 


1.  Given  an  undirected  network  edge  {u,  u},  we  will 
often  refer  to  it  as  a  directed  edge  contextually  in 
the  direction  of  signaling,  i.e.,  along  e  =  {u,  v),  u 
exports  path  descriptors  to  v.  We  write  e'  =  (u,  u) 
for  the  edge  in  the  opposite  direction  (forwarding  or 
import). 

2.  The  class  of  an  edge  e  =  {v,  u)  is  c(e)  =  C'^{u). 

3.  Let  x(Ci)  =  {Cj  I  Xij  =  1}.  Then  for  any  edge  e, 
c(e')  G  x(c(e)). 

4.  Let  m(C'i)  =  {Cj  \  Mij  =  1}  and  let  m.~^{Ci)  = 
{C,  I  Mp  =  1}. 

So,  m(C'i)  is  the  set  of  classes  to  which  a  path  descrip¬ 
tor  learned  from  a  neighbor  of  class  Ci  can  be  exported 
without  an  increase  in  level,  and  is  the  set  of 

classes  from  which  a  path  descriptor  exported  to  a  neigh¬ 
bor  of  class  Ci  without  an  increase  in  level  could  have 
been  imported. 

Definition  3.6  (Homogeneity  and  Heterogeneity).  A 

dispute  wheel  is  homogeneous  of  type  Ck  iff  for  all  edges 
e  along  the  rim,  c(e)  =  Cfc.  A  dispute  wheel  is  hetero¬ 
geneous  of  types  Cki ,  Ck2 , ...  iff  for  all  edges  e  along  the 
rim,  c(e)  €  {Cfe, ,  , . . .}. 


PVPS  solvability,  and  thus  checking  robustness,  is  VP- 
hard  in  general  [6];  we  will  thus  try  to  find  an  easy-to- 
check  global  constraint  in  the  class-based  context  which 
rejects  as  few  robust  instances  as  possible.  Prop.  3.4  and 
the  matrices  in  a  class  description  together  restrict  the 
edge  types  (and  pairs  of  edge  types)  which  may  appear 
on  the  rim  of  a  dispute  wheel.  Because  Thm.  3.3  tells  us 
that  precluding  dispute  wheels  guarantees  robustness  for 
all  instances,  we  can  use  those  restrictions  to  produce  a 
sufficient  global  constraint. 

Call  the  rim  nodes  at  the  ends  of  paths  Ri  in  a  dispute 
wheel  active  nodes',  these  are  the  nodes  at  which  route 
preferences  cause  the  dispute  wheel.  In  Fig.  1,  node  v  is 
an  active  node  with  neighboring  rim  nodes  u  and  x — these 
may  or  may  not  be  active  nodes  themselves.  The  neighbor 
on  the  spoke  is  w,  the  dispute  occurs  because  the  route 
through  u  is  preferred  over  the  direct  route  through  w,  but 
the  route  from  w  is  (or  has  been)  exported  to  x.  Note  that 
edges  in  Fig.  1  have  arrows  in  the  direction  of  signaling 
or  export. 

Prop.  3.4  tells  us  that  the  level-attribute  values  of 
dispute-wheel  paths  are  the  same  at  the  rim,  so  the  matrix 
M  must  permit  level  equality  for  the  class  assignments  on 
a  dispute-wheel  rim.  The  matrix  W  must  also  permit  the 
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preferences  required  by  Def.  3.2(5)  (a  necessary  condition 
for  a  dispute  wheel).  In  particular,  any  of  the  following 
conditions  at  one  active  node  would  preclude  the  dispute 
wheel  in  Fig.  1;  they  are  written  from  the  perspective  of 
active  node  v  without  loss  of  generality: 

(Cl)  V  could  not  import  a  descriptor  from  w  and  export 
it  to  X  without  increasing  level,  i.e.,  Mp-y  =  0; 

(C2)  V  must  prefer  routes  from  w  over  those  from  u,  i.e., 
Waf3  =  1;  or 

(C3)  u  could  not  export  a  descriptor  from  a  neighbor 
and  export  it  to  v  without  increasing  level,  i.e., 
Vv,gc  =  0;  etc. 

The  above  conditions  could  be  achieved  by  forcing  the 
class  assignments  a,  /3,  7,  "0,  and  C,  to  specific  values. 
Note  that  in  (Cl)  and  (C3),  a  particular  pair  of  class  as¬ 
signments  (/3  and  7,  or  ijj  and  Q  is  troublesome  because 
of  the  associated  M-matrix  entry.  This  idea  of  pairs  of  as¬ 
signments  will  be  used  later  to  describe  the  general  global 
constraint. 

We  now  prove  a  result  based  on  the  discussion  above. 
The  following  gives  conditions  that  are  necessary  for  an 
edge  to  participate  in  a  dispute-wheel  rim  (i.e.,  it  general¬ 
izes  (Cl)  and  (C3)  above). 

Proposition  3.7  (Rim  Necessary  Condition).  Let  edge  e 
be  on  the  rim  of  a  dispute  wheel.  If  c[e)  =  Ca,  then 

1.  3Ci3  G  C  :  Mfja  =  1;  i-e.,  m”^(C'a)  f  0;  and 

2.  3(7^,  e  C,  €  x(m“^(C'a))  :  =  1. 

Proof.  (Sketch.)  Condition  1  follows  directly  from 
Prop.  3.4.  Condition  2  follows  directly  from  Lemma  8.6 
in  the  extended  version  of  [5].  □ 

If,  for  each  class  Ca,  either  one  of  these  conditions  does 
not  hold  or,  if  both  do  hold,  no  edges  of  class  Ca  are  al¬ 
lowed,  then  all  instances  will  be  dispute-wheel  free.  This 
was  the  constraint  suggested  in  [5],  but  it  is  unreason¬ 
ably  strong  because  it  precludes  many  robust  instances; 
in  particular,  the  role  of  condition  (C2)  is  ignored,  and 
that  could  break  dispute  wheels  in  some  instances. 

Relying  on  condition  (C2) — tweaking  preferences 
locally — is  not  enough,  however;  e.g.,  if  the  class  assign¬ 
ment  to  both  incoming  edges  is  the  same,  no  system-wide 


rule  can  prevent  the  dispute.  We  now  give  four  negative 
results  in  this  regard:  each  shows  the  existence  of  a  sim¬ 
ple  instance  containing  a  dispute  wheel,  given  a  certain 
combination  of  entries  for  just  one  or  two  classes  in  the 
matrices  X  and  M.  These  are  canonical  instances  that 
can  be  generalized.  All  but  one  of  the  cases  do  not  require 
specific  values  in  matrix  W  for  the  construction;  this  sug¬ 
gests  that  while  the  constraint  in  [5]  is  too  strong,  some 
constraint  preventing  pairs  of  class  assignments  will  be 
necessary. 

First  we  introduce  some  notation:  because  matrix  M 
involves  the  scoping  rule  between  an  import  and  export, 
the  class  assignments  used  to  look  up  values  in  M  are  not 
all  in  the  same  direction  (that  of  signaling).  We  define 
a  matrix  S  incorporating  the  matrices  M  and  X:  entry 
Sij  =  1  if  a  descriptor  can  be  exported  along  two  sig¬ 
naling  edges,  first  of  class  Ci,  then  of  class  Cj,  without 
increasing  the  level  attribute.  Equivalently,  we  may  de¬ 
fine  S  in  terms  of  X  and  M  as  follows. 

Definition  3.8.  S  =  XM  (boolean),  i.e.,  Sij  = 
min((XM),„l)  e  {0,1}. 

We  now  proceed  to  the  negative  results.  The  first  neg¬ 
ative  result  involves  the  existence  of  dispute  wheels  with 
only  one  type  of  class  assignment  on  the  rim.  One  of  the 
cases  requires  certain  values  in  W  for  the  construction; 
the  other  case  does  not. 

Proposition  3.9  (Existence  of  Homogeneous  Dispute 
Wheels).  Suppose  Ca  G  C.  If  either 

1.  Saa  =  1,  or 

2.  there  exists  some  Cp  €  m.~^(Ca)  such  that  f 
—  1  for  some  C.y  €  x(Ca), 

then  there  exists  an  instance  that  contains  a  homogeneous 
dispute  wheel  of  type  Ca. 

Proof.  For  case  1,  we  can  construct  a  simple  dispute 
wheel  where  all  spokes  and  rim  segments  have  length  one, 
and  for  any  of  these  edges  e,  c(e)  =  Ca  and  c(e')  =  Cp 
for  the  same  class  Cp  €  x(C'q,).  Then,  regardless  of  W,  it 
is  possible  for  every  rim  node  to  export  a  spoke  descriptor 
to  its  neighbor  because  Saa  =  1  and  for  every  rim  node 
to  prefer  the  neighbor’s  path  because  the  spoke  and  rim 
neighbors  are  assigned  the  same  class  and  we  must  have 
Wpp  f  -1. 
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For  case  2,  we  can  construct  a  similar  dispute  wheel, 
but  while  c(eji)  =  Ca  for  rim  edges  cr,  we  have  that 
c(eQ)  =  (7/3  for  the  reverse  spoke  edges  e'g.  Let  c(e^)  = 
for  7  as  defined  in  the  proposition  statement.  Then,  re¬ 
gardless  of  W,  it  is  possible  for  every  rim  node  to  export  a 
spoke  descriptor  to  its  neighbor  because  (7/3  €  ni~^(Ca) 
and  for  every  rim  node  to  prefer  the  neighbor’s  path  be¬ 
cause  —  1-  □ 

Example  3.10  (Homogeneous  Dispute  Wheels  in 
HBGP).  For  the  system  in  Ex.  2.2,  we  have 


S  = 


10  0 
1  0  0 
1  1  1 


so  homogeneous  dispute  wheels  of  type  customer  or 
provider  are  possible.  Thus,  as  shown  in  [3,  5,  6],  in¬ 
stances  without  customer-provider  cycles  are  robust. 

The  next  three  results  show  how  to  construct  dispute 
wheels  with  two  types  of  class  assignments  on  the  rim, 
regardless  of  W .  None  of  these  results  require  particular 
values  in  W  for  the  existence  of  the  dispute  wheels. 

Proposition  3.11  (Simple  Heterogeneous  Dispute 
Wheel).  If  there  exist  CcnCf}  G  (7  such  that  Sa/s  = 
S/3a  =  1.  there  exists  an  instance  containing  a  het¬ 
erogeneous  dispute  wheel  of  types  Ca,  Cp. 

Proof  We  can  create  a  dispute  wheel  with  origin  uq  and 
two  rim  nodes  ui,U2.Let  =  Ca,C'"°{v2)  =  (7/3, 

C'’^{v2)  =  (7/3,  and  =  Ca-  Then  it  is  possible 

for  vi  to  assign  the  same  class  in  x((7q,)  to  vq  and  V2  and 
prefer  the  rim  path  over  the  spoke  path;  similarly  for  V2 
with  x((7/3).  The  export  from  spoke  to  rim  is  possible 
precisely  because  Sap  =  Spa  =  1.  and  no  setting  in  W 
can  break  this  dispute  wheel.  □ 


x((7q,)  n  x((7/3)  0,  it  is  possible  for  vi  and  V2  to  assign 

the  same  class  C-y  G  x((7a)  (7  x{Cp)  to  vq  and  the  rim 
neighbor.  Because  Wyy  7^—1,  both  can  prefer  the  rim 
path  over  the  spoke  path.  The  export  from  spoke  to  rim 
is  possible  precisely  because  Saa  =  Spp  =  1,  and  no 
setting  in  W  can  break  this  dispute  wheel.  □ 

Proposition  3.13  (Reverse  Heterogeneity).  If  there  exist 
Ca,  Cp  €  (7  such  that  3Cy  G  x(Ca)  with  Mpy  =  1  and 
3(7,/,  G  x(Cp)  with  Maip  =  L  then  there  exists  an  in¬ 
stance  containing  a  heterogeneous  dispute  wheel  of  types 

Cy,  C-ijj. 

Proof  We  can  create  a  dispute  wheel  with  origin  vq  and 
two  rim  nodes  vi,V2-  Let  C^^  (vq)  =  Ca,  C’'^  (vq)  =  Cp, 
C'’'^{v2)  =  Ca,  and  (7"^(t;i)  =  Cp.  These  assignments 
are  in  the  reverse  direction  of  export  along  the  wheel. 
But,  given  the  statement  of  the  proposition,  we  can  set 
C'^°{vi)  =  Cy,  (7"«(u2)  =  C^,  (7"^(ui)  =  Cy,  and 
(7”^  (^2)  =  Cjjj  and  obtain  a  heterogeneous  dispute  wheel 
of  types  Cy,  Cijj  similar  to  the  dispute  wheel  constructed 
in  the  proof  of  Prop.  3.11.  □ 

The  constructions  in  the  above  proofs  can  be  extended 
to  larger  dispute  wheels,  possibly  involving  more  class 
types,  as  long  as  the  X-,  M-,  and,  in  some  cases,  W- 
matrix  entries  permit.  We  now  construct  a  predicate 
P  on  classes  which  generalizes  conditions  (C1)-(C3). 
¥{Ci,Cj)  will  be  true  exactly  when  nodes  u,  v,  and  w 
may  be  part  of  a  dispute  wheel  in  which  u  is  a  rim  node, 
V  imports  from  u  and  exports  to  w,  and  (7"  (u)  =  (7/  and 
C^iw)  =  Cj.  Note  that  from  the  proofs  above,  we  see 
that  just  two  permitted  rim  nodes  are  necessary  for  some 
instance  to  contain  a  dispute  wheel. 

Definition  3.14  (Dispute  Predicate).  Let  P  be  a  predicate 
on  two  classes 


Proposition  3.12  (Non-permutation  Heterogeneity). 

Suppose  X  is  not  a  permutation  matrix.  If  there  ex¬ 
ist  Ca,Cp  G  C  such  that  Saa  =  L  spp  =  1,  and 
x((7q)  n  x((7/3)  7^  0,  then  there  exists  an  instance  con¬ 
taining  a  heterogeneous  dispute  wheel  of  types  Ca,  Cp. 

Proof  We  can  create  a  dispute  wheel  with  origin  uq  and 
two  rim  nodes  vi,V2.  Let  C'"°{vi)  =  Ca,  C'’°{v2)  = 
Cp,  C'“^(v2)  =  Ca,  and  C'"^{vi)  =  Cp.  Then  because 


nCa,Cp)  ^ 

{{Map  =  1)  V  {3Cy  G  m~\Cp)  :  Wya  f  -l))  . 

The  claim  that  P  is  the  condition  we  want  is  supported 
by  the  following  theorem.  It  states  that  a  cycle  where  P 
holds  pairwise  along  the  edges  could  be  a  dispute  wheel; 
and,  if  all  cycles  where  P  holds  pairwise  along  the  edges 
are  prevented,  no  dispute  wheel  can  exist. 
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Theorem  3.15  (Exact  Condition  for  Dispute  Wheel 
Creation).  For  1  <  i  <  n,  let  ki,k^  £  {1, . . . ,  c}  io  that 
CkijCj-i'  £  C;  let  kn+i  =  ki.  There  exists  an  instance 
containing  a  dispute  wheel  with  rim  cycle  ei ,  62 , . . . ,  e„, 
where  e„  is  adjacent  to  ei,  c(ei)  =  and  c(e^)  =  C}~>., 
P(C'fc/,  is  true. 

Proof.  We  first  prove  the  forward  “only-if”  direction.  As¬ 
sume  there  exists  an  instance  containing  such  a  dispute 
wheel.  Let  Vi  be  the  node  incident  to  edges  and  ei+i; 
its  class  assignments  for  the  neighbor  incident  to  Ci  and 
ti+i  are  then  c(e-)  =  Ck'.  and  c(ei+i)  =  re¬ 

spectively.  Every  rim  node  Vi  is  either  active  (one  with 
a  direct  spoke  path)  or  inactive  (within  a  rim  segment).  If 
it  is  inactive,  then  some  path  descriptor  imported  along 
Ci  must  be  exported  along  e^+i  without  an  increase  in 
level  (by  Prop.  3.4);  thus,  Mk'ki+i  =  C  which  implies 
'P{Ck'.,Cki+i).  If  it  is  active,  then  the  imported  spoke 
path  is  exported  along  e^+i  without  increasing  level  (by 
Prop.  3.4);  thus,  it  must  assign  the  neighboring  spoke 
node  w  a  class  O'"*  (w)  =  C.y  such  that 

=  1.  (1) 

Furthermore,  the  descriptor  imported  along  must  be 
preferred  more  than  the  spoke  path;  thus,  it  must  be  that 

W^k,,  ^ -I  (2) 

so  that  the  rim  preference  is  allowed.  (1)  and  (2)  together 
imply  P(C'fe' ,  ).  By  considering  every  node  Vi  in  this 

way,  we  see  that  P(Cfc' ,  )  must  hold  for  all  i. 

In  the  other  direction,  we  construct  the  specified  dispute 
wheel  if  P(C'fe' ,  holds  for  all  i.  Build  a  cycle  of 

edges  ei,  62, . . . ,  e^,  with  e„  adjacent  to  ei,  and  assign 
c(ei)  =  Cki,  c(e')  =  Ck'.  Assume  there  is  a  destination 
node  d.  As  PjC^' ,  holds,  either  Mk’.k^^^  =  1  or 

e  ^ -1.  (3) 

First  assume  that  Mk'.ki+i  =  1;  in  this  case,  the  node  be¬ 
tween  Ci  and  Ci+i  can  be  left  an  inactive  node.  If  (3)  is 
true,  then  the  node  Vi  between  and  ej+i  can  be  made 
an  active  node;  in  this  case,  connect  the  destination  node 
d  to  Vi  and  let  C'^'{d)  =  C,y  such  that  Cj  satisfies  (3). 
Then  let  node  Vi  prefer  the  route  imported  from  the  rim 
along  Ci  over  the  route  from  the  spoke  along  (d,  Vi).  This 


is  permitted  because  W.yk’.  —  1  by  (3).  We  note  that 

we  can  choose  at  least  two  nodes  Vi  to  be  active  nodes — 
the  minimum  required  for  a  dispute  wheel — because  even 
if  Mk'.ki+i  =  1,  then  we  can  set  Vi  to  be  an  active  node 
connected  to  d  with  c{vi,d)  =  c(e').  If  both  descrip¬ 
tors  imported  at  Vi  are  from  neighbors  of  the  same  class, 
then  the  rim  path  can  be  preferred  over  the  spoke  path, 
which  is  enough  to  cause  the  dispute.  Therefore,  this  cy¬ 
cle  of  edges  ei,  62, . . . ,  e„  with  destination  d  and  class 
assignments  set  as  indicated  forms  a  dispute  wheel  that  is 
permitted  by  the  class  description.  □ 

Thm.  3.15  identifies  our  robustness  constraint  exactly: 
to  prevent  dispute  wheels  in  any  network,  we  must  check 
against  all  cycles  where  P  holds  on  all  pairs  of  edges  in 
the  cycle;  if  these  cycles  are  permitted,  they  are  potential 
dispute  wheels.  Formally,  we  have  the  following. 

Constraint  3.16  (Class-Based  Robustness).  For  all  cy¬ 
cles  of  signaling  edges  ei,  62, . . . ,  e„,  there  exists  some 
i,  1  <  i  <  n  such  that  F  {c{e'f),c{ei+i))  does  not  hold 
(assume  that  e„+i  =  ei). 

By  Thm.  3.3,  instances  obeying  this  constraint  are  robust. 
Because  the  presence  of  a  dispute  wheel  does  not  preclude 
solvability,  we  cannot  say  that  this  constraint  is  tight. 

Algorithm  3.17  (Generation  of  Robustness  Con¬ 
straint).  Given  a  class  description,  we  can  find  the  “trou¬ 
blesome”  pairs  of  assignments  satisfying  P  in  O(c^)  time, 
where  c  is  the  number  of  classes,  using  the  following 
naive  procedure: 

1.  For  all  1  <  z,  j  <  c,  examine  Wij.  If  Wij  ^  —  1, 
create  a  list  T'  of  classes  in  m(Ci).  Add  the  pair 
(j,  t)  for  all  t  £  T'  to  the  list  T. 

2.  For  all  1  <  z,  j  <  c,  examine  Mij.  If  My  =  1  then 
add  the  pair  {i,j)  to  T. 

Note  that  for  any  pair  (fi,  ^2)  €  T,  P(Ctj ,  Ct^)  holds. 

4  Centralized  Dispute- Wheel 
Prevention 

Although  we  can  now  identify  the  constraint  for  a  given 
class  description,  we  have  not  yet  mentioned  how  to  en- 
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force  this  condition.  In  this  section,  we  describe  a  cen¬ 
tralized  algorithm  that  operates  on  an  instance  graph  and 
detects  violations  of  Constraint  3.16. 

4.1  Cycle-Detection  Algorithm 

Given  an  instance  with  undirected  network  G  =  {V,E) 
and  nodes’  class  assignments,  we  want  to  identify  all  cy¬ 
cles  in  which  P  holds  on  all  consecutive  pairs  of  edges 
around  the  cycle.  We  do  this  as  follows. 

Algorithm  4.1  (Centralized  Cycle  Detection). 

1.  Construct  a  digraph  Gs  =  {V,Es)  using  the  same 
vertices  as  in  the  network  G.  For  every  edge  in 
{u,  u}  G  E,  Es  contains  the  directed  edges  (u,  v) 
and  (u,  u). 

2.  Construct  a  new  digraph  Gl  =  (Cl,  El)  from  Gs 
as  follows.  Let  Vl  =  Eg-  El  contains  an  edge  from 
(m,  v)  to  (w,x)  iff  V  =  w  and  P(c(u,  m),  c(i;,  a;)) 

holds. 

3.  Do  a  directed  depth-first  search  of  Gl-  Any  directed 
cycles  found  correspond  to  potential  dispute  wheels. 

Proposition  4.2  (Correctness).  Any  cycle  in  Gl  found 
by  Alg.  4.1  corresponds  to  a  potential  dispute  wheel  in  the 
original  network  and  every  dispute  wheel  in  the  network 
produces  a  directed  cycle  in  Gl- 

Proof  Let  eo,  ei, . . . ,  e*,  be  a  directed  cycle  in  Gl,  with 
Bi  =  {{ui,Vi),  for  Ui,Vi  €  V  andvi  =  Ui+i 

for  0  <  i  <  k  (subscripts  modulo  k  -f  1).  By  construc¬ 
tion  of  Gl,  P(c(t;i,  Ui),  Vi+i))  holds  for  every  i,  so 
the  cycle  {uo,Vo},{ui,Vi},  - .  -  ,{uk,Vk}  violates  Con¬ 
straint  3.16  {Le-,  it  is  a  potential  dispute  wheel). 

By  Prop.  3.4,  the  rim  {uo,  t^i},  {t^i, '^2},  ■  ■  • ,  t^o} 

of  a  dispute  wheel  in  G  is  such  that 
P(c(?;i, Ui-i), 0(1;,, Ui+i))  holds  for  all  i.  Thus  El 
contains  {vi,Vi+i))  for  every  i,  producing  a 

directed  cycle  in  Gl  -  □ 

Proposition  4.3  (Running  Time).  Alg.  4.1  has  running 
time  0(A|£’|)  on  a  network  G  =  (C,  E)  with  maximum 
vertex  degree  A. 


Proof.  Construction  of  G5  takes  0(|C|  -f  |i5|)  time,  con¬ 
struction  of  Gl  takes  0(|yL|  +  \El\)  =  0{\E\  +  A|i?|) 
time,  and  cycle  checking  Gl  by  depth-first  search  takes 
0(|^l|  +  \El\)  =  0{\E\  H-  A|£’|)  time.  Therefore,  the 
total  running  time  is  O  ( A  |  i?  | ) .  □ 

4.2  Checking  Next-Hop  Preferences 

Suppose  we  have  a  network  running  a  path-vector  pro¬ 
tocol  for  which  each  node  v  specifies  a  partial  order  < 
on  neighbors  such  that  for  two  neighbors  u  and  w,  if 
u<iw,  then  routes  imported  from  u  must  be  ranked  lower 
{i.e.,  more  preferred)  than  routes  imported  from  w,  and 
if  u  =  u>,  then  no  relative  preference  is  forced  between 
routes  imported  from  u  and  w.  Furthermore,  we  allow 
nodes  to  describe  scoping  rules  for  these  neighbors  (un¬ 
der  what  conditions,  if  any,  routes  can  be  exported).  These 
policies  are  called  next-hop  preferences,  because  the  rel¬ 
ative  preference  and  scoping  rules  for  routes  are  deter¬ 
mined  by  the  next  hop  along  the  path,  i.e.,  the  neighbor 
from  which  the  path  descriptor  is  imported. 

Given  a  network  and  next-hop  preferences,  we  can  con¬ 
struct  a  class-based  system  consistent  with  the  nodes’  rel¬ 
ative  preference  and  scoping  rules.  Define  a  class  descrip¬ 
tion  in  which  there  is  a  class  for  every  directed  signal¬ 
ing  edge  in  the  network,  and  assign  every  neighbor  the 
class  corresponding  to  the  edge  between  the  node  and  that 
neighbor.  Then  set  the  entries  of  W  to  be  consistent  with 
the  partial  order  for  next-hop  preferences,  and  set  the  en¬ 
tries  of  M  to  be  consistent  with  the  scoping  rules.  Most 
of  the  entries  of  W  and  M  are  irrelevant  because  not  all 
edges  in  the  graph  are  adjacent,  so  comparisons  will  never 
have  to  be  made  between  all  possible  pairs  of  classes. 
Essentially,  each  node’s  next-hop  preferences  define  sub¬ 
matrices  of  W  and  M. 

By  creating  this  consistent  class-based  system,  we  can 
use  the  robustness  checks  developed  in  this  paper  to  see 
whether  this  network  with  its  next-hop  preferences  has 
any  potential  dispute  wheels.  Because  there  is  a  class  for 
every  directed  edge  in  the  signaling  graph,  there  will  be 
c  =  2\E\  classes,  and  generation  of  the  constraint  pairs 
satisfying  P  using  Alg.  3.17  will  take  O(c^)  =  0{\E\^) 
time.  However,  not  all  pairs  of  classes  correspond  to  ad¬ 
jacent  edges;  assuming  that  A  is  the  maximum  degree  of 
any  vertex  in  the  graph,  each  node  in  V  gives  less  than  A^ 
pairs  of  directed  signaling  edges.  For  each  pair  we  must 
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check  one  entry  in  M  and  at  most  A  in  W,  so  P  may 
actually  be  generated  in  0(A^|y|)  time.  Running  the 
centralized  cycle-check  (Alg.  4.1)  takes  0{A\E\)  time. 
The  total  time  is  thus  0(A^|y|  +  A|ii^|),  but  because 
\E\  =  0(A|V^|),  the  running  time  is  simply  0(A^|y|). 

Below  we  give  the  full  algorithm  that  executes  the 
constraint-generating  and  centralized  constraint-checking 
procedures  with  the  running  time  discussed  above  for  a 
network  G  =  (V,  E)  with  next-hop  preferences  (relative 
preference  and  scoping  rules  at  each  node  for  all  neigh¬ 
bors  of  that  node). 

Algorithm  4.4  (Check  for  Next-Hop  Preferences). 

1.  Construct  a  set  T  C  V^.  For  every  node  v  G  G, 
repeat  the  following  for  each  neighbor  x  of  v:  For  all 
neighbors  u  7^  a;  of  u,  if  a  descriptor  at  v  imported 
from  u  can  be  exported  to  x  (without  level  increase), 
then: 

(a)  add  the  triple  (u,  u,  x)  to  the  set  T;  and 

(b)  for  all  neighbors  w  such  that  w  <  u,  add  the 
triple  (u,  w,  x)  to  the  set  T. 

After  iterating  through  all  nodes  v  and  pairs  of  neigh¬ 
bors  X  and  u,  note  that  elements  of  the  set  T  cor¬ 
respond  to  pairs  of  class  assignments  satisfying  the 
predicate  P  in  the  next-hop-preferences  sense. 

2.  Construct  a  new  directed  graph  Gl  =  {Vl,  El). 
The  vertex  set 

Vl  =  {(m,  v)  and  {v,  u)  \  {u,  v}  G  E} ; 

i.e.,  there  is  one  vertex  for  every  directed  signaling 
edge.  The  edge  set 

El  =  {((m,  ■y),  {v,  x))  I  (u,  u,  x)  GT}- 

i.e.,  there  is  a  directed  edge  from  {u,  v)  to  {v,  w) 
iff  these  two  signaling  edges  can  be  adjacent  on  a 
dispute-wheel  rim. 

3.  Cycle-check  Gl  using  directed  depth-first  search. 
Any  directed  cycles  found  correspond  to  potential 
dispute  wheels. 


Proposition  4.5  (Correctness  of  Next-Hop-Preferences 
Check).  Any  cycle  in  Gl  found  by  Alg.  4.4  corresponds 
to  a  potential  dispute  wheel  in  the  original  network  and 
every  dispute  wheel  in  the  network  produces  a  directed 
cycle  in  Gl. 

Proof.  The  graph  Gl  checked  for  cycles  in  Alg.  4.4  is 
similar  to  the  graph  Gl  checked  for  cycles  in  Alg.  4.1; 
the  difference  is  that  the  edges  are  based  on  next-hop- 
preference  conditions  instead  of  a  predefined  W  and  M. 
Therefore,  this  result  follows  from  Prop.  4.2  if  we  show 
that  {v,  u,  x)  G  T  iff  P(c(u,  u),  c{v,  x))  would  hold  for 
the  W  and  M  created  from  the  next-hop  preferences. 

However,  this  is  simple  to  derive  from  the  construc¬ 
tion  of  T  in  step  1  of  Alg.  4.4:  (u,  u,  x)  G  T  iff 
(1)  either  an  import  over  (it,  v)  can  be  exported  over 
{v,  x)  without  level  increase,  thus  Mc(v,u)  c(v,x)  =  1; 
or  (2)  there  exists  some  neighbor  y  of  u  such  that  it  <  y 
and  a  descriptor  learned  from  y  can  be  exported  to  x 
without  a  level  increase;  thus  3y  G  m”^(c(ii, a:))  : 
^c{v,y)  c(v,u)  f  ~1-  These  conditions  are  exactly  equiv¬ 
alent  to  P(c(i;,  it),  c(ii,  x)).  □ 

4.3  Algorithms  in  Previous  Work 

Sobrinho  [12]  presents  an  algebraic  formalism  for  path- 
vector  routing  that  is  shown  in  [9]  to  be  essentially 
equivalent  to  PVPSes.  Sec.  6.3  in  [12]  gives  a  check 
for  protocol  convergence  on  a  network  given  the  ab¬ 
stract  design  specification  of  the  protocol — much  like 
our  centralized  algorithm  given  the  constraint  generated 
from  the  class  description.  Translated  to  the  class-based 
framework,  the  class-aware  constraint-generating  algo¬ 
rithm  and  convergence-checking  algorithm  from  [12]  take 
time  O(c^)  and  0(c  ■  (|y|  -f  |i?|)),  where  c  is  the  num¬ 
ber  of  classes,  assuming  that  matrix  W  is  consistent  with 
a  linear  order  on  G.  The  performance  of  this  algorithm 
compared  to  Alg.  4.1  will  depend  on  how  the  number 
of  classes  c  compares  to  the  vertex  degrees  in  a  network 
and  how  sparse  the  network  is.  Also  note  that  the  algo¬ 
rithm  from  the  algebra  framework  might  give  some  false 
positives:  it  identifies  some  cycles  as  troublesome  that 
are  not  actually  potential  dispute  wheels  {i.e.,  the  con¬ 
straint  checked  is  stronger  than  necessary  and  stronger 
than  Constraint  3.16).  We  discuss  this  difference  below 
and  show  how  to  implement  this  stronger,  but  sometimes 
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faster,  convergence  check  in  our  framework. 

In  the  algebra  framework,  routing  policy  is  captured  by 
the  assignment  of  a  label  to  each  edge  in  the  signaling 
graph.  Changes  to  the  attributes  of  a  path  descriptor  when 
it  is  shared  between  neighboring  nodes  are  modeled  by 
an  operation  that  depends  on  (1)  the  label  on  the  signal¬ 
ing  edge  between  the  neighbors,  and  (2)  the  signature  of 
the  path  before  it  is  shared — this  corresponds  to  the  orig¬ 
inal  path  descriptor;  the  result  is  a  new  signature,  or  path 
descriptor,  that  can  be  ranked  at  the  importing  node.  Be¬ 
cause  the  algebra  framework  does  not  separately  model 
import  and  export  transformations,  the  label  assigned  to 
the  edge  must  capture  the  policy  decisions  made  at  both 
import  and  export.  In  the  class-based  framework,  policy 
decisions  depend  on  class  assignments  between  neigh¬ 
bors.  Therefore,  when  using  the  algebra  framework  to 
model  a  class-based  system,  the  labels  on  signaling  edges 
must  capture  the  class  assignment  in  both  directions — 
both  nodes’  view  of  the  other  node — in  order  to  capture 
the  relative-preference  and  scoping  rules  that  eventually 
affect  the  rank  or  availability  of  path  descriptors.  Thus,  if 
c  is  the  number  of  classes,  there  are  possible  labels. 

The  robustness  algorithm  in  [12]  first  generates  a  set  of 
labels  with  which  to  check  cycles  (this  corresponds  to  the 
generation  of  our  constraint  involving  pairs  of  classes): 
it  identifies  sets  of  labels  that  come  from  pairs  of  la¬ 
bels  satisfying  an  equivalent  notion  of  P.  (The  variable 
w  indexes  these  sets,  ordering  them  based  on  the  relative- 
preference  order  of  the  labels  falling  into  the  sets.)  The 
algorithm  then  uses  these  sets  to  check  the  freeness  con¬ 
straint:  it  checks  the  graph  for  cycles  formed  by  edges 
whose  labels  all  belong  to  one  of  these  sets  (for  some 
value  w).  Because  the  sets  are  missing  information 
that  could  be  used  to  detect  that  some  cycles  would  not 
actually  be  dispute  wheels,  some  instances  are  flagged  as 
problematic  even  though  they  are  robust. 

The  following  scenario  causes  the  algorithm  to  pro¬ 
duce  a  false  positive.  We  will  refer  to  labels  with  a  pair 
of  classes  indicating  the  class  assignments  in  both  direc¬ 
tions  along  a  signaling  edge.  Suppose  that  (Cq.,  Ca>)  and 
(C/3, (7/3')  is  the  only  pair  of  labels  involving 
that  satisfies  the  equivalent  notion  of  P  {i.e.,  this  pair  of 
edge  types  could  be  on  a  dispute-wheel  rim).  The  algo¬ 
rithm  in  [12]  will  add  (C/3,  C/3')  to  the  appropriate  set 
Lu,.  Suppose  that  the  label  (C-,,,  C^')  also  belongs  to  L^, 
because  it,  too,  is  part  of  a  pair  of  labels  satisfying  the 


equivalent  notion  of  P.  The  algorithm  would  then  remove 
all  cycles  in  which  all  edge  labels  belong  to  L^.  Consider 
such  a  cycle  with  two  adjacent  edges  labeled  {C^,C^i) 
and  {Ci3,C(}i).  It  may  be  the  case  that  ^  1,  ie., 

these  edges  could  not  actually  participate  in  a  dispute  be¬ 
cause  doing  so  would  violate  class-consistency.  The  stor¬ 
age  of  labels  in  basically  throws  away  one  half  of  the 
pair  satisfying  P,  in  this  case,  the  label  {Ca,Ca')-  As  a 
result,  cycles  that  could  not  have  class  consistency  on  the 
overlapping  edges  and  be  dispute  cycles  at  the  same  time 
are  still  flagged. 

Algorithm  4.6  (Algebraic  Robustness  Check,  adapted 
from  [12]).  Steps  1-3  generate  the  algebraic  robustness 
constraint,  the  sets  L^-,  steps  4-5  check  that  constraint. 

1 .  By  assumption,  W  is  consistent  with  a  linear  order 
<C  on  the  classes  C.  To  each  class  Ci  assign  a  value 
w{Ci)  G  {1,  . . . ,  c}  such  that  w{Ci)  <  w{Cj)  if 
Ci  <c  Cj.  Let  w*  =  max/  w{Ci)',  thus  w*  =  0(c). 

2.  Use  Alg.  3.17  to  generate  pairs  of  classes  on  which 
P  holds.  This  takes  O(c^)  time. 

3.  For  each  w,  1  <  w  <  w*,  construct  the  set  of  labels 

i»  =  {((7/3,C/3')e(7x(7|3C'„eC': 

P((7q,,  (7/3)  A  A/3/3'  =  1  a  w{Ci3')  =  u>}. 

This  takes  (7(c^)  time  because  the  sets  can  be  built 
by  examining  the  0{(f)  pairs  ((7a,  (7/3)  satisfying  P 
and,  for  each,  examining  the  (7(c)  classes  (7/3'  so  that 
if  A/3/3'  =  1,  ((7/3,  C/3')  is  added  to  the  set 

4.  Given  the  network  G  =  (U,  E),  construct  the  graph 
Gs  =  (U,  Es)  as  in  Alg.  4.1.  Then  for  each  w, 
I  <  w  <  w*,  construct  the  graph  Gs[w]  =  (U,  E^) 
where  e  G  E^  iff  c(e)  =  C/3,  c(e')  =  C/3'  such  that 
(C/3,  C/3')  G  Ljju.  This  takes  0(c|£'|)  time,  total. 

5 .  Cycle-check  each  Gs  [w] ;  any  cycle  is  a  potential  dis¬ 
pute  wheel.  This  takes  0(c  •  (|U|  +  |7?|))  time  by 
depth-first  search. 
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5  Distributed  Dispute- Wheel 
Prevention 

Although  the  Internet  graph  and  node  relationships  do  not 
change  haphazardly,  a  centralized  algorithm  running  on  a 
snapshot  of  the  Internet  graph  is  still  somewhat  infeasible: 
A  central  source  would  need  to  collect  information  about 
the  network  topology  as  well  as,  in  a  potentially  harder 
and/or  privacy-invading  task,  information  about  node  re¬ 
lationships  throughout  the  network.  In  this  section,  we 
first  present  a  distributed  algorithm  for  detecting  potential 
dispute  wheels  and  then  contrast  this  algorithm  with  one 
given  in  earlier  work. 

5.1  Distributed  Cycle-Check 

Our  distributed  algorithm  (Alg.  5.1)  detects  potential  dis¬ 
pute  wheels  that  include  a  specified  edge  on  their  rim. 
The  algorithm  is  administered  by  the  two  nodes  connected 
by  the  edge  in  question;  it  sends  at  most  three  messages 
across  each  edge  in  the  graph  and  does  not  require  that  the 
graph,  minus  the  edge  in  question,  is  dispute-wheel-free. 
Furthermore,  the  algorithm  reveals  little  about  the  rela¬ 
tionships  between  nodes  in  the  graph — a  node  may  learn 
possibilities  for  its  neighbors’  relationships  with  other 
nodes,  but  nothing  about  other  relationships  in  the  graph. 

If  the  algorithm  detects  the  edge  as  problematic,  either 
the  edge  can  be  removed  from  the  signaling  graph  {i.e., 
the  edge  is  not  used  to  advertise  routes)  or  some  tweaks 
to  local  policy  can  be  applied  to  prevent  a  dispute  wheel. 
These  tweaks  are  included  in  Alg.  5.1  and  allow  the  edge 
to  exist  as-is  for  the  purpose  of  signaling  routes  that  would 
never  cause  a  dispute.  This  algorithm  could  be  ran,  e.g., 
by  two  nodes  before  adding  a  signaling  link  to  the  Inter¬ 
net  graph  to  see  what  policy  tweaks  must  be  enforced  to 
prevent  route  oscillations. 

In  summary,  node  u  starts  the  algorithm  by  sending  out 
a  forward  token  [N,  F]  to  u.  is  a  nonce,  which  prevents 
interference  between  parallel  executions  of  the  algorithm, 
and  F  indicates  that  this  is  a  (forward)  token.  Any  node  w 
along  the  way,  including  v,  that  receives  this  token  from 
some  node  x  passes  a  copy  of  the  token  along  to  a  neigh¬ 
bor  y  if  P(c(r(;,  x),  c{w,  y))  holds  and  w  has  not  already 
forwarded  the  token  to  y.  In  this  way,  the  token  traverses 
all  pairs  of  edges  that  could  be  part  of  a  potential  dispute 


wheel.  If  a  cycle  of  edges  is  traversed,  i.e.,  u  receives 
its  starting  token  [N,  F]  and  would  forward  it  to  v,  then 
u  knows  that  the  edge  (it,  v)  participates  in  a  potential 
dispute-wheel  rim.  Token-traversal  paths  end  when  there 
are  no  neighbors  y  that  should  be  forwarded  the  token; 
in  that  case,  a  receipt,  or  “backwards”  message,  [N,  B]  is 
sent  to  the  neighbor  from  whom  the  token  was  received.  If 
a  node  w  receives  receipts  from  all  neighbors  to  whom  it 
forwarded  the  token,  w  then  sends  a  receipt  to  the  neigh¬ 
bor  from  whom  it  received  the  token.  Note  that  only  one 
forward  token  needs  to  be  sent  along  any  edge;  any  dupli¬ 
cate  tokens  sent  along  an  edge  will  take  the  same  route  as 
the  original  token,  and  this  has  no  effect  on  cycle  detec¬ 
tion  from  It’s  perspective.  We  thus  know  that  all  token- 
traversal  paths  will  terminate — in  the  worst  case,  after  the 
token  has  traversed  every  edge  once.  The  algorithm  essen¬ 
tially  ends  when  u  receives  a  receipt  from  v,  indicating 
that  all  token  traversals  have  ended.  Node  u  then  sends 
out  an  all-clear  message  [N,C],  which  gets  forwarded 
along  the  token-traversal  paths,  so  that  other  nodes  can 
delete  any  data  structures  used  for  that  instance  of  the  al¬ 
gorithm.  Once  the  algorithm  has  ended,  if  u  detected  a 
problem,  then  u  can  either  refuse  to  signal  along  {u,v)  or 
tweak  policies  so  that  a  dispute  wheel  could  never  form 
along  the  cycle. 

Algorithm  5.1  (Distributed  Edge  Check).  A  node  u 
should  start  the  following  procedure  to  check  the  signal¬ 
ing  edge  (m,  u);  when  checking  the  network  edge  {u,  u}, 
V  should  separately  check  the  signaling  edge  {v,  u)  in  the 
opposite  direction.  Assume  that  nodes  have  a  list  Lq  for 
storing  nonces  from  different,  parallel  executions  of  this 
algorithm.  Let  the  in-neighbors  of  v  be  denoted  in(u)  and 
the  out-neighbors  be  denoted  out(u). 

For  node  u: 

1 .  Choose  and  store  a  nonce  N.  Create  an  empty  list  of 
nodes  Lb. 

2.  If  3u>  S  in(u)  such  that  P(c(m,  w),  c{u,v))  holds, 
then  u  sends  the  message  [N,  F]  to  v  along  (u,  v). 

3.  Whenever  u  receives  [/V,  F]  from  w  G  in(u),  send 
the  message  [N,B]  to  w.  If  P(c(u,  w),  c{u,v)) 
holds,  then  add  the  node  w  to  the  list  Lb. 

4.  When  u  receives  the  message  [N,  B]  from  v,  u 
should  send  [N,  C]  to  v.  Node  u  may  now  start 
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routing  along  (u,  v)  after  applying  the  appropriate 
policy-tweak  rules  below  to  nodes  in  list  Lb- 

5.  Node  u  should  ignore  any  [N,  C]  messages. 

For  all  nodes  w  ^  u: 

1.  If  ru  receives  the  message  [TV,  F]  from  x  G  in(ii;): 

(a)  If  ^  Lq,  add  N  to  list  Lq  and  create  an 
array  An  of  lists  of  type  {V  x  {0, 1})  indexed 
by  the  elements  of  in(?ii). 

(b)  For  each  y  G  out(r(;),  if  P(c(u>,  a:),  c{w,y)) 
and  (y,0),  (y,  1)  ^  An{z)  for  all  z  G  in(rt;), 
then  send  [N,  F]  to  y  and  add  (y,  0)  to  An{x). 

(c)  If  no  [TV,  F]  messages  were  sent  above  in  step 
(b),  then  send  [A^,  B]  to  x. 

2.  If  w  receives  [AT,  C]  and  N  G  Lq,  then  for  each 
z  G  in(w),  send  [AT,  C]  to  each  y  such  that  (y,  1)  G 
An{z).  Delete  Am  and  remove  N  from  Lq. 

3.  If  w  receives  [AT,  B]  from  y  G  out(w),  then  replace 
(y,  0)  with  (y,  1)  in  Am-  If  ((y,  i)  G  Am^x)  (i  = 
1)),  i.e.,  if  y  is  the  last  node  in  the  list  Am(x)  from 
which  [AT,  B]  was  received  for  some  x  G  in(w),  then 
send  [AT,  B]  to  x. 

The  following  are  the  policy-tweak  rules  for  u  if,  at  the 
end  of  the  algorithm,  is  not  empty.  Let 

Y'^{w)  =  {y  G  in(u)  |  G  m~^(C'“(u)) 

A  ILc“(y)C“(u.)  ^  -1  }• 

Node  u  should  depreference  routes  from  w  G  Lb  with 
respect  to  all  y  G  V^iw).  This  is  only  possible  iff 

t{w,  w')  :  {w'  G  Y^{w))  /\{wG  Y^{w'))  (4) 

because  two  neighbors  cannot  both  be  depreferenced  with 
respect  to  each  other.  If  (4)  does  not  hold,  then: 

1.  Pick  some  w  G  Lb  ■  Y'^{w)  ^  0.  For  all  w'  ^  w, 
if  Y'^iyo')  ^  0,  then  increase  the  level  attribute  on 
import  from  w'  and  remove  vJ  from  L b  ■ 

2.  Depreference  w  with  respect  to  all  other  y  G  Y'^{w). 

3.  For  all  w  G  Lb  remaining,  increase  the  level  on 
routes  imported  from  w  when  exported  to  v. 


The  following  propositions  assert  various  properties  of 
the  algorithm,  including  correctness. 

Lemma  1  (Number  of  Tokens).  In  Alg.  5.1,  at  most  one 
[AT,  F]  token  is  sent  along  each  signaling  edge. 

Proof.  For  every  node  w  u,?in  [AT,  F[  message  is  only 
sent  to  a  neighbor  if  an  [AT,  F]  message  is  received,  but 
the  [AT,  F]  message  is  not  sent  if  the  neighbor  has  already 
received  an  [AT,  F]  message  from  w,  regardless  of  how 
many  [AT,  F]  messages  are  received  at  w.  Because  this 
process  starts  with  u  sending  one  [AT,  F]  message  to  v,  it 
is  clear  that  at  most  one  [AT,  F]  message  is  sent  between 
every  pair  of  nodes.  □ 

Proposition  5.2  (Termination).  The  algorithm  termi¬ 
nates,  i.e.,  every  node  that  sends  to  y  or  receives  from 
X  a  message  [AT,  F]  receives  from  y  or  sends  to  x,  respec¬ 
tively,  a  message  [AT,  B]. 

Proof  We  must  show  that  if  the  [AT,  F]  token  is  sent  along 
some  edge  {x,  y),  then  an  [AT,  B]  receipt  is  sent  back  from 
y  to  X.  Consider  a  graph  Gt  =  (Vt,  Ft)  constructed 
from  the  target  network  G  =  (V,  E).  The  vertex  set  Vt 
is  the  set  of  directed  signaling  edges  of  G,  and  there  is  an 
edge  in  Et  from  {x,  y)  to  {y,  z)  if  the  receipt  of  a  token 
[AT,  F]  at  y  from  x  causes  y  to  send  the  token  [AT,  F]  to  z. 

Note  that  the  connected  component  of  Gt  is  a  tree 
rooted  at  (u,  u);  this  is  because  Lem.  1  tells  us  that  an 
[AT,  F]  token  is  sent  at  most  once  along  a  signaling  edge, 
and  we  can  see  from  the  algorithm  that  an  [AT,  F]  token  is 
only  sent  by  a  node  after  receiving  one  itself.  Therefore 
every  node  in  Gt  has  either  no  ancestors  (if  the  node  is 
{u,  v)  or  [AT,  F]  is  never  sent  along  the  edge)  or  exactly 
one  ancestor. 

First  consider  the  leaf  nodes  of  the  tree;  these  are  edges 
(x,  y)  such  that  receipt  of  the  [AT,  F]  token  at  y  does 
not  cause  an  [AT,  F]  token  to  be  sent.  According  to  the 
algorithm,  this  happens  in  two  cases:  (1)  y  =  u;  or 
(2)  there  does  not  exist  a  neighbor  z  of  y,  for  which 
P(c(?/,x),  c{y,z))  holds,  that  has  not  already  received 
a  token.  In  both  of  these  cases,  an  [AT,  B]  message  is  sent 
back  to  X  from  y. 

Next  consider  the  ancestors  of  leaf  nodes  in  the  tree. 
Given  the  argument  above,  we  know  that  for  such  an  edge 
(x,  y),  node  y  receives  an  [AT,  B]  message  from  all  neigh¬ 
bors  z  such  that  {y,  z)  is  a  descendant  of  (x,  y).  Accord¬ 
ing  to  the  algorithm,  once  this  has  happened,  every  entry 
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in  the  list  Aiy{x)  at  y  will  be  of  the  form  (z,  1);  thus  y 
will  send  an  [N,  B]  receipt  to  x.  This  argument  can  be 
repeated  for  the  ancestors  of  these  nodes,  etc.,  so  that  all 
tokens  are  eventually  acknowledged  with  receipts. 

The  [N,  C]  messages  terminating  the  algorithm  follow 
the  path  of  the  tree,  so  that  every  node  that  initially  re¬ 
ceived  the  token  will  destroy  any  data  associated  with  the 
nonce  N.  □ 

Proposition  5.3  (Cycle  Participation).  At  the  end  of  the 

algorithm,  if  Lb  f  0  at  u,  then  (u,  v)  is  part  of  a  cycle 
violating  Constraint  3.16. 

Proof.  If  Lb  f  0,  then  there  exists  some  w  &  Lb  such 
that  w  sends  u  the  message  [iV,  f].  This  means  that 
u  originated  the  message  [iV,  F],  sending  it  to  v,  and 
there  is  a  set  of  nodes  {u  =  Xi,  X2,  •  ■  ■ ,  Xn  =  tu} 
such  that  every  node  Xi  sends  [iV,  F]  to  (assume 
Xn+i  =  u).  According  to  the  algorithm,  this  only 
happens  if:  (1)  P(c(m,  w),  c{u,v))  holds;  and  (2)  if 
F{c{xi,Xi-i),  c{xi,Xi+i))  holds  for  all  1  <  z  <  n.  We 
then  have  a  cycle  of  edges 

(it,  V),  (V,  X2),  (X2,  X3),  ...,  (x„-i,  W),  (W,  It) 

where  P  holds  pairwise  along  adjacent  edges;  this  is  a 
potential  dispute  wheel  containing  (it,  v).  □ 

Proposition  5.4  (Number  of  Messages).  The  algorithm 
sends  either  0  or  3  messages  per  signaling-graph  link. 

Proof.  By  Lem.  1,  at  most  one  [W,  F]  message  is  sent 
along  a  link.  If  an  [N,  F]  message  is  sent,  it  is  clear  from 
the  algorithm  and  the  proof  of  Prop.  5.2  that  one  [N,  B] 
message  and  one  [N,  C]  message  are  sent  along  the  link, 
and  that  no  other  messages  are  generated  as  a  result  of  the 
[N,  F]  message.  Therefore,  the  total  number  of  messages 
sent  along  a  given  link  is  either  0  (no  [N,  F]  is  sent)  or  3 
([N,  F],  [N,  B]  and  [N,  C]  are  sent).  □ 

Depending  on  the  structure  of  the  graph,  a  token- 
traversal  path  might  include  all  the  edges  in  a  graph;  but, 
in  this  case,  this  will  be  the  only  token-traversal  path  (be¬ 
cause  tokens  are  only  sent  once  per  edge). 

Alg.  5.1  preserves  privacy  in  the  following  ways.  As 
the  messages  involved  contain  only  a  nonce  and  message 
type,  the  edge  being  checked  by  a  run  of  the  algorithm 


is  not  revealed  to  nodes  other  than  u.  Furthermore,  be¬ 
cause  Prop.  5.2  tells  us  that  every  [W,  F]  message  sent 
is  acknowledged  with  an  [N,  i?]-message  reply,  nothing 
in  the  algorithm  tells  any  of  the  other  nodes  whether  or 
not  a  potential  dispute  wheel  has  been  detected — only  u 
knows  this.  The  only  information  learned  is  that  if  a  node 
w  receives  an  [N,  F]  message  from  x,  it  knows  that  there 
exists  some  neighbor  z  of  a;  such  that  P(c(a:,  z),  c(a;,  w)) 
holds,  w  might  then  narrow  down  the  possibilities  for  the 
assignments  C'^(z)  and  C^{w),  although  the  latter  is  al¬ 
ready  restricted  by  C'^(x)  and  the  matrix  X.  Node  w 
does  not  learn  any  other  information  about  nodes’  class 
assignments. 

There  is  an  inherent  trade-off  between  the  number  of 
messages  sent  by  the  algorithm  and  the  state  retained  at 
each  node.  Alg.  5.1  can  be  modified — without  sacrificing 
privacy — to  delete  the  state  for  nonce  N  early,  instead  of 
waiting  for  an  [N,  C]  message.  There  are  two  possible 
modifications: 

1.  The  list  Ajv(z)  can  be  deleted  once  an  [N,  B]  mes¬ 
sage  is  sent  to  in-neighbor  z;  or 

2.  The  array  of  lists  Ajv  can  be  deleted  once  [N,B] 
messages  are  sent  to  all  in-neighbors  z  such  that 
Ajsi^z)  is  nonempty,  i.e.,  once  all  tokens  have  been 
acknowledged. 

While  these  modifications  eliminate  the  need  for  the 
[N,  C]  message,  they  lose  some  benefit  of  aggregating 
token-traversal  paths.  Consider  a  node  w  that  has  ac¬ 
knowledged  all  tokens  with  receipts.  Using  either  mod¬ 
ification,  w  now  retains  no  state  for  this  run  of  the  algo¬ 
rithm.  However,  it  could  receive  the  forward  token  from  a 
neighbor  that  had  not  yet  sent  it  a  token  {e.g.,  because  of 
network  delays)  or  from  a  neighbor  that  previously  sent 
it  a  token  {e.g.,  because  it  deleted  state  as  well).  As  a  re¬ 
sult  of  either  of  these  (and  certainly  in  the  latter  case),  w 
might  send  a  token  to  a  neighbor  it  had  previously  sent  a 
token,  thus  duplicating  a  token-traversal  path  and  increas¬ 
ing  the  total  number  of  messages  used  by  the  algorithm. 
Given  the  first  modification,  this  could  happen  even  when 
there  are  outstanding  tokens  that  have  not  been  acknowl¬ 
edged.  Therefore,  these  modifications  may  be  appropriate 
in  certain  networks  where  sending  duplicate  tokens  is  un¬ 
likely  and  in  networks  where  maintaining  state  or  send¬ 
ing  the  [N,  C]  message  is  expensive;  however,  in  most 
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cases,  these  modifications  will  result  in  duplicating  mes¬ 
sages  along  token-traversal  paths  with  little  added  benefit. 

5.2  Algorithms  in  Previous  Work 

The  algorithm  SPVP3  in  [8]  is  a  distributed  path-vector 
routing  algorithm  that  detects  local-policy-based  routing 
oscillation  while  running.  SPVP3  essentially  adds  a  path- 
history  attribute  to  path  descriptors;  it  stores  the  changes 
in  best-route  choices  that  cause  the  descriptor  to  be  adver¬ 
tised.  If  there  is  a  cycle  in  these  changes,  then  the  descrip¬ 
tor  being  advertised  is  contributing  to  a  route  oscillation 
due  to  local  policies.  These  path-history  cycles  are  shown 
in  [8]  to  correspond  exactly  to  dispute  wheels.  Therefore, 
in  the  process  of  routing,  SPVP3  detects  the  actual  policy 
conflicts  forming  a  dispute  wheel. 

The  algorithms  in  this  paper  take  a  different 
approach — they  attempt  to  detect  and/or  prevent  poten¬ 
tial  dispute  wheels  before  a  routing  dispute  ever  occurs. 
But  more  importantly,  the  idea  of  including  a  constraint  as 
part  of  the  system  specification  allows  us  to  prove  proper¬ 
ties  about  the  system  at  design-time.  The  centralized  and 
distributed  algorithms,  then,  essentially  implement  con¬ 
straint  enforcement  rather  than  find  modified  solutions  on- 
the-fly.  The  class-based  path-vector  routing  protocol  will 
work  as  expected  as  long  as  the  system  has  been  designed 
to  prevent  bad  policy  interactions. 

One  final  difference  is  that  SPVP3  essentially  removes 
the  rim  paths  on  a  dispute  wheel  from  the  choices  that  all 
rim  nodes  have  for  paths  to  a  destination.  This  is  unnec¬ 
essary,  because  the  dispute  wheel  only  needs  to  be  “bro¬ 
ken”  at  one  active  node  by  tweaking  preferences.  SPVP3 
prevents  multiple  nodes  (rather  than  one)  from  being  as¬ 
signed  its  more-preferred  path,  whereas  our  distributed  al¬ 
gorithm  tweaks  policies  at  one  node  to  correct  a  potential 
dispute  wheel.  It  is  also  worth  noting  that  SPVP3  re¬ 
quires  potentially  large  routing  messages  (the  length  of 
the  path  history  will  be  on  the  order  of  the  size  of  the 
dispute-wheel  rim).  Also,  the  entire  detection  process 
might  be  repeated  for  the  same  cycle  and  slightly  mod¬ 
ified  spoke  paths  (or  a  different  destination),  whereas  the 
cycle-detection  algorithms  will  detect  or  fix  a  potential 
dispute-wheel  rim  before  it  is  used  for  routing,  so  that  this 
cycle  does  not  cause  oscillation  for  any  destination  or  set 
of  spoke  paths.  The  downside  of  this,  however,  is  that 
some  policy  tweaks  or  filtering  might  be  used  to  fix  the 


potential  dispute-wheel  rim  even  if  no  route  oscillation  ac¬ 
tually  occurs,  whereas  SPVP3  deals  with  the  oscillations 
as  they  happen  dynamically  without  affecting  policies  for 
other  routes. 

6  Conclusion  and  Future  Work 

In  this  paper,  we  have  extended  previous  work  on  path- 
vector  policy  systems  by  focusing  on  class-based  systems, 
which  generalize  Hierarchical-BGP  and  related  protocols. 
In  particular,  we  show  how  to  use  the  specification  of  a 
generic  class-based  system  to  generate  a  global  constraint 
which  guarantees  the  robust  convergence  of  any  network 
instance  satisfying  it.  Our  constraint  is  the  best-known 
such  constraint  for  these  systems,  and  we  provide  cen¬ 
tralized  and  distributed  algorithms  to  enforce  it.  How¬ 
ever,  our  constraint  is  not  likely  to  be  the  most  general, 
tractable  constraint  for  all  path-vector  systems.  To  date, 
class-based  systems  seem  to  be  the  only  path-vector  sys¬ 
tems  well-characterized  enough  that  exact  constraints  for 
them  can  be  proven,  but  studying  other  characterizations 
may  yield  constraints  and  enforcement  mechanisms  that 
guarantee  robustness  for  a  larger  set  of  path-vector  proto¬ 
cols.  In  addition,  the  question  of  how  to  efficiently  run  our 
distributed  algorithm  in  parallel  remains  open:  in  partic¬ 
ular,  token-traversal  paths  from  separate  instances  of  the 
algorithm  can  probably  be  combined  to  find  and  fix  all 
potential  dispute  wheels  at  once. 
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